Papers with neural topic model
Extracting Topics with Simultaneous Word Co-occurrence and Semantic Correlation Graphs: Neural Topic Modeling for Short Texts (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical results validate that DWGTM can generate more semantically coherent topics than baseline topic models. |
| Approach: | They develop a neural topic model which extracts topics from word co-occurrence graphs . Empirical results validate that DWGTM can generate more semantically coherent topics than baseline topic models. |
| Outcome: | Empirical results show that the proposed model can generate more coherent topics than baseline topic models. |
Topic-Guided Abstractive Multi-Document Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies on multi-document summarization (MDS) focus on extractive and abstractive approaches to create a fluent and concise summary for a collection of thematically related documents. |
| Approach: | They propose a novel abstractive MDS model that represents multiple documents as a heterogeneous graph and then applies a graph-to-sequence framework to generate summaries. |
| Outcome: | The proposed model outperforms state-of-the-art models on Rouge scores and human evaluation, while learning high-quality topics. |
Improving Neural Topic Models using Knowledge Distillation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current paradigms for transfer learning use general knowledge as a foundation for more specialized endeavors. |
| Approach: | They propose to combine probabilistic topic models and pretrained transformers to improve topic quality by using knowledge distillation. |
| Outcome: | The proposed framework improves topic quality over all estimated topics and in head-to-head comparisons of aligned topics. |
Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder (2020.emnlp-main)
Copied to clipboard
| Challenge: | Topic models for short texts suffer from data sparsity because of limited word co-occurrences. |
| Approach: | They propose a neural topic model with a new topic distribution quantization approach that generates peakier distributions that are more appropriate for modeling short texts. |
| Outcome: | The proposed model outperforms both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics. |
A Disentangled Adversarial Neural Topic Model for Separating Opinions from Plots in User Reviews (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing topic models may extract topics associated with writers’ subjective opinions mixed with those related to factual descriptions. |
| Approach: | They propose a neural topic model combined with adversarial training to disentangle opinion topics from plot and neutral ones. |
| Outcome: | The proposed model shows improved coherence and variety of topics, consistent disentanglement rate, and superior sentiment classification performance to other supervised topic models. |
Self-Supervised Neural Topic Modeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Topic models are useful tools for analyzing and interpreting the main underlying themes of large corpora of text. |
| Approach: | They propose a self-supervised neural topic model that learns a topic representation jointly from three co-occurring words and a document that the triple originates from. |
| Outcome: | The proposed model outperforms existing topic models in coherence metrics and document clustering accuracy. |
Multilingual and Multimodal Topic Modelling with Pretrained Embeddings (2022.coling-1)
Copied to clipboard
| Challenge: | a novel neural topic model for comparable data maps texts from multiple languages and images into a shared topic space. |
| Approach: | They propose a novel multimodal multilingual neural topic model that maps texts from multiple languages and images into a shared topic space. |
| Outcome: | The proposed model outperforms a zero-shot topic model in predicting topic distributions for comparable multilingual data and performs as well on unaligned embeddings as it does on aligned embeds. |
CWTM: Leveraging Contextualized Word Embeddings from BERT for Neural Topic Modeling (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing topic models rely on bag-of-words (BOW) representations to capture word order information. |
| Approach: | They propose a neural topic model that integrates contextualized word embeddings from BERT to learn the topic vector of a document without BOW information. |
| Outcome: | The proposed model generates more coherent and meaningful topics compared to existing models while accommodating unseen words in newly encountered documents. |
Pre-training and Fine-tuning Neural Topic Model: A Simple yet Effective Approach to Incorporating External Knowledge (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that using external knowledge such as pre-trained word embeddings or pre-train language models only achieved limited performance improvements but with huge computational overhead. |
| Approach: | They propose to incorporate external knowledge into neural topic modeling by pre-trained word embeddings (PWEs) or pre-train language models (PLMs) they propose to fine-tune the neural topic model on the target dataset and reduce the huge size of training data. |
| Outcome: | The proposed approach outperforms current state-of-the-art neural topic models and some topic modeling approaches enhanced with PWEs or PLMs on three datasets and greatly reduces the huge size of training data. |
TITA: A Two-stage Interaction and Topic-Aware Text Matching Model (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models for keyword-document matching do not define topic-aware relevance clearly. |
| Approach: | They propose a two-stage interaction and topic-aware text matching model to solve this problem . they propose to combine latent topic of document with deep neural representation to model complex interactions between keyword and document. |
| Outcome: | The proposed model outperforms other well-designed baselines and shows excellent performance in the recommendation system. |
#HowYouTagTweets: Learning User Hashtagging Preferences via Personalized Topic Attention (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods based on latent topics cannot capture user interests and thus can't be used to predict how likely a user will post with a hashtag. |
| Approach: | They propose a personalized topic attention model that captures salient contents to personalize hashtag contexts by predicting how likely a user will post with a hashtag. |
| Outcome: | The proposed model significantly outperforms the state-of-the-art recommendation approach without exploiting latent topics. |
Topic Modeling with Wasserstein Autoencoders (P19-1)
Copied to clipboard
| Challenge: | Existing probabilistic topic models are based on latent Dirichlet allocations and collapsed Gibbs sampling. |
| Approach: | They propose a novel topic model that enforces Dirichlet prior on latent document-topic vectors and a kernel kernel to minimize the Maximum Mean Discrepancy (MMD) They propose to measure the diversity of the produced topics and to use the widely used coherence measure NPMI to evaluate topic quality. |
| Outcome: | The proposed model performs better than existing topic models on real datasets. |
Enriching and Controlling Global Semantics for Text Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization models have been proven effective in creating fluent and informative summaries, but they suffer from the short-range dependency problem, causing them to produce summary that miss the key points of document. |
| Approach: | They propose a neural topic model empowered with normalizing flow to capture global semantics of the document and integrate them into the summarization model. |
| Outcome: | The proposed model outperforms state-of-the-art summarization models on five common text summarizing datasets, namely CNN/DailyMail, XSum, Reddit TIFU, arXiv, and PubMed. |
Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing topic seed words are difficult to incorporate into topic models due to the semantic diversity of natural language. |
| Approach: | They propose a neural topic model enhanced with supervisions from seed words on word and document levels. |
| Outcome: | The proposed model outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy. |
Semantics-Aware Dual Graph Convolutional Networks for Argument Pair Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Argument pair extraction (APE) aims to extract interactive argument pairs from two separate passages. |
| Approach: | They propose to tackle the lexical and semantic relevance of arguments with a pre-trained Rouge-guided Transformer (ROT) by using a word graph and a gating mechanism to fuse two graphs. |
| Outcome: | The proposed approach achieves state-of-the-art on F1 score and significantly improves on existing best alternative. |
TopicDiff: A Topic-enriched Diffusion Approach for Multimodal Conversational Emotion Detection (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies focus on learning contextual information in conversations, neglecting acoustic and vision topic information. |
| Approach: | They propose a model-agnostic Topic-enriched Diffusion approach for capturing multimodal topic information in MCE tasks. |
| Outcome: | The proposed approach improves over the state-of-the-art MCE models and the existing models. |